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Abstract 

In this note we consider the Ky Fan fc-norm matrix conic optimization problem, 
which includes the nuclear norm regularized minimization problem as a special case. 
For this class of nonpolyhedral matrix conic optimization problems, we show that 
the solution mappings of two types of perturbed KKT systems are locally upper 
Lipschitz at the origin for a KKT point, under the second-order sufficient condition of 
the stationary point and the strict Robinson’s CQ for the associated multiplier. This 
result depends on an equivalent characterization for the point pair in the graphical 
derivative of the normal cone mapping of the fc-norm matrix cone, and implies a local 
error bound which plays a crucial role in the convergence rate analysis of algorithms. 

Keywords: matrix conic optimization; perturbed KKT system; locally upper Lip¬ 
schitz; second-order sufficient condition; strict Robinson’s CQ 
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1 Introduction 

Throughout this note, we write X = M x with m < n, where is the vector 

space of all m X n real matrices endowed with the trace inner product (•, •) and its induced 
Frobenius norm || • ||. Let K := |(t, A) £ X | ||(T(A)||(fc) < t} be the Ky Fan fe-norm 
matrix cone for an integer k € [1, nr], where cr{X) G MX denotes the singular value vector 
of X with nonincreasing entries, and || • means the vector Ky Fan /c-norm. Given 
twice continuously differentiable /: X —)• M, /i: X —)• and G: X —)• X, we are interested 
in the following Ky Fan fc-norm matrix conic optimization problem 

min {/(A): /i(R') = 0, G(R') G A} , (1) 
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which is well-known to have a host of applications. A typical example is the popular 
nuclear norm regularized least squares problem (see, e.g., [3, 14, 8, 4]) of the form 

rnin lh\A{X)-bf+p\\X\U: S{X) = d] (2) 

where A: —)■ RP and S: R™^"- —)• R? are the linear mappings, 6 G RP and d G R'^ 

are the vectors, and p > 0 is the regularizing parameter. Clearly, one may rewrite (2) as 
the form (1) with k = m, f{X) = ^||^(X) — 6|p -|- pt, h{X) = £{X) — d and G{X) = X 
for X = {t,X). For other special examples, the interested reader may refer to [5]. 

Let L:XxRPxX—)-R denote the Lagrange function of problem (1), defined by 

L(A; A, J) := f{X) + {h{X), A) + {G{X),y) 

where A G R^ and 3^ G X are the Lagrange multipliers associated to the equality con¬ 
straint h{X) = 0 and the conic constraint G{X) G K, respectively. It is known that the 
Karush-Kuhn-Tucker (KKT) optimality conditions for problem (1) take the form 

L'^.(A’;A,T) = 0, /i(A) = 0 and T G A/'x(G(A’)) (3) 

where is the derivative of L with respect to A, and Mk{G{X)) is the normal cone of 
K at G{X) in the sense of convex analysis. By the definition of A//f, it is easy to obtain 

K 3 G{X) xy ^ Mk{G{X)) ^ G{X) - IIk{G{X) + T) = 0, (4) 


where K° is the negative polar cone of K, and : X —>• X is the projection operator 
associated to K. So the KKT optimality conditions in (3) can be equivalently written as 


/ L',(T,A,T) \ 

T(T,A,T):= h{X) =0. (5) 

\G{X)-YiK{G{X)+y)l 


In the sequel, for a given feasible point T G X of problem (1), we denote by Xi{X) the 
set of Lagrange multipliers, and say that T is a stationary point of problem (1) if and 
only if A4(A’) 7^ 0. It is well known that if T is a locally optimal solution to problem 
(1), then X may not be a stationary point. If Robinson’s constraint qualihcation (CQ) 
holds at X, then Ai(X) A 0 ^ind T is a stationary point. When T is a stationary point, 
we say that a Lagrange multiplier (A,T) G Xi{X) satisfies the strict Robinson’s CQ if 


(h'{PQ\ 

\G'{X)) 


X + 


i 1 

\TK{G{x))r^y^ 



( 6 ) 


where Tk{G{X)) is the contingent cone of K at G{X), and T~*" := {Z G X : {Z,y) = O}. 
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This note is mainly concerned with the solution mappings of the perturbed KKT 
systems T(T’, X,y) = 6 and T((5, X, A, T) =0 for 6 = {6f, 6h, Sg) G X x x X, where 

/ L'^{X,X,y)-6f \ 

T(,5,T',A,T):= hiy)-6h • (7) 

\G{y) -5g- n^(G(Af) -6G + y)J 

Notice that T(h, Al, A, T) = 0 is the KKT system of the following perturbation of (1) 

h{y) = 6^, G{X) e 6 g + k}, ( 8 ) 

whereas T(A:',A,T) = 5 does not correspond to the KKT system of any perturbation of 
(1). Let j7:XxM^xX^XxM^xX and ^TrXxM^xX^XxRPxXbe the solution 
mappings associated to the above two types of perturbed KKT systems, respectively, i.e.. 


J{5) := {(7f,A,T) GXxM^xX 

1 T(A’,A,T) =h} 

(9) 

J(6) := {(A’,A,T) GXxRPxX 1 

T(h,Af,A,T) =0}. 

(10) 


Clearly, {X, A, T) is a KKT point of (1) iff (0, X, A, T) G gphj" or gph J", while {X, A, T) 
is a KKT point of (8) associated to h = {df,Sh, Sc) G X x x X iff (5, X, A, T) G gph J'. 

The main contribution of this work is to establish the locally upper Lipschitz of the 
multifunctions J and ^7 at 0 for >V = {X, A,T) G J7(0) = 77(0), under the second-order 
sufficient condition of X and the strict Robinson’s CQ for (A,T)- As will be shown in 
Remark 3.1(b), the locally upper Lipschitz of 77 at 0 implies that the distance from any 
point (X, A, T) near >V to the whole set of KKT points can be controlled by the KKT 
system residual at this point. This local error bound plays a key role in achieving the 
convergence rate of the first-order algorithms for (1), and especially the nuclear norm 
regularized least-squares problem. While the locally upper Lipschitz of 77 is important 
in the perturbation theory of optimization. This is the main motivation of this work. 

We notice that Zhang and Zhang [17] recently derived the locally upper Lipschitz of 
the KKT mapping for the canonical perturbation of nonlinear semidehnite programming 
(SDP) problems by the equivalent Kojima’s reformulation, under the second-order suffi¬ 
cient condition and the strict Robinson’s CQ. Later, Han, Sun and Zhang [9] established 
the locally upper Lipschitz of the perturbed KKT system for the nonlinear SDP problem 
under the same assumption and used it to provide a sufficient condition to guarantee 
the linear convergence rate of the ADMM (alternating direction method of multipliers) 
for the convex composite quadratic SDP problem. This note is also motivated by their 
works and the wide applications of the Ky Fan A:-norm matrix conic optimization. When 
making revisions for our manuscript, we learn that Ding, Sun and Zhang [7] provide an 
equivalent characterization for the isolated calmness (i.e., the locally upper Lipschitz) of 
the KKT solution mapping for a large class of conic optimization problems. 

To close this section, we present a brief summary for the notations used in this paper. 
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• Let S™ be the vector space of all m x m real symmetric matrices, and the 

set of all m X A: real matrices with orthonormal columns. We simplify as 

O™. For any Z E a{Z) denotes the singular value vector of Z whose entries 

are arranged in a nonincreasing order, and for any Z S S™, X{Z) is the eigenvalue 
vector with decreasing entries. For Z £ 0™’’"'(Z) means the following set 

Qm,n^Z) := e O™ X O” I Z = U\Bmg{a{Z)) . (11) 

Similarly, if Z G S™', then 0^(Z) denotes the set {P G | Z = PDiag(A(Z))P'’^}. 

• For a closed set S and a point x £ S, Tgi^) and Ts{x) denote the inner tangent cone 
and the contingent cone, respectively, and J\fs(x) means the limit normal cone of S 
at X. When S is convex, J\fs{x) is the normal cone in the sense of convex analysis. 

• The e, E and I denote a vector, a matrix of all entries being 1 and the unit matrix, 
respectively, whose dimensions are known from the context. For a given Z G 

and an index set J C {1,..., n}, Zj is an mx | J| matrix consisting of those columns 
Zj with j G J, and for x G ffi”, xj G is a vector consisting of Xi with i £ J. 

2 Preliminaries 

In this section, Y and Z denote the finite dimensional vector spaces equipped with the 
norm || • ||. For a multifunction 5: Y ^ Z, two important sets associated with it are the do¬ 
main dom5 := {?/ G Y I 5(y) / 0} and the graph gph5 := {(y, z) G Y x Z | z G <S(y)}. 
First, we recall the locally upper Lipschitz (see [10, 12]) of a multifunction at a point. 

Definition 2.1 A multifunction 5: Y ^ Z is said to be locally upper Lipschitz at y for 
'z £ S{y) if there exist a constant fj. > 0 and neighborhoods U of y and V of z such that 

Vn5(y) = {z} and 5(y) n V C {j} + ^||y - ypy ^y £U, 

where By denotes the unit ball in the space Y centered at the origin. 

The locally upper Lipschitz concept of 5 at y for z £ S{y) in Definition 2.1 is different 
from the one defined by Robinson [15], which actually requires iS(y)nV is upper Lipschitz 
at y. The locally upper Lipschitz of 5 at y for z £ S{y) is also called the isolated calmness 
of 5 at y for z G S{y) in [9]. By Lemma 1 in Appendix A, we have the following equivalent 
characterization for the locally upper Lipschitz of a multifunction. 

Lemma 2.1 The locally upper Lipschitz of a multifunction 5: Y ^ Z aty for 'z £ S{y) 
is identified with the existence of a constant y > 0 and a neighborhood V of z such that 

Vn5(y) = {z} and 5(y) n V C {z} +/i||y - ypY Vy G Y. 

In addition, from [13, 10] we also have the following equivalent characterization for 
the locally upper Lipschitz property of a multifunction at a point of its graph. 
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Lemma 2.2 Let 5: Y ^ Z he a multifunction. Then S is locally upper Lipschitz at y 
forz G S{y) if and only if DS{y\z){0) = {0}. 

Next we recall the graphical derivative of a multifunction 5: Y ^ Z from [16, 8G]. 

Definition 2.2 Consider a multifunction 5 : Y ^ Z and a point y G dom5. The 
graphical derivative of S aty for any z^S{y) is the mapping DSify\z) : Y ^Z defined by 

V £ DS{y\z){u) {u,v) £Tgphs{y,z). 

When S is single-valued at y, we simplify the notation DSify\z) to be DS{y). 

If the multifunction 5: Y ^ Z is implicitly defined by a (directionally differentiable) 
single-valued mapping, then we have the following result for its graphical derivative. 

Lemma 2.3 Let 5: Y ^ Z be a multifunction defined by S{y) := {z G Z | F{y, z) = 0} 
where F:Y x h ^ h is a single-valued mapping. Consider any (y, z) G gphiS. Then, 

D5(ylz)(u) C |u G Z : 0 G DF(y, z)(u, u)| Vu G Y. 

If F is directionally differentiable at {y,'z) G gph5, then it holds that 

DS(fy\z){u) C |u G Z : F'{{y,z)-, {u,v)) = o| Vtt G Y. 

In particular, when F{y, z) := y Il{z) for any y G Y and z G Z with if: Z —)• Z being 
a single-valued mapping, the above two inclusions become equalities. 

Proof: Consider any u G Y and v G DS{fy\'z){u). Then, {u,v) G 7^ph5(y)^)- By the 
dehnition of 7^ph<s(y)^)! there exist sequences ^ u,v^ ^ v and in i 0 such that 

z + tnv'^ £ S{y + tnu'^) F(fy-£ tnU^,ztnv'^) = 0 for all n. 

This shows that there exist sequences re"' = 0, {u,v) and tn i 0 such that 

tnw'^ = F(jj -\-tnU^ ,'z -\-tnv"^) for all n. Consequently, ((u, u),0) G 7^phF((y)^)) 0); he., 
0 G DF(jI,z){u,v). The first inclusion follows by the arbitrariness of v in iiiS(y|z)(u). 
When F is directionally differentiable at (y, z) G gph5, 0 G DF{y,z){u,v) is equivalent 
to F'{{y,'z)‘, (u, v)) = 0 (see [12, Equation(6.6)]), and the second inclusion follows. For the 
last part, it suffices to prove that G Y: 0 G DF{y,z){u, u)} C DS{y\z){u). Indeed, let 
V be an arbitrary point such that 0 G DF{y,z){u,v). Then ((u, u),0) G 7^phF((y)T), 0), 
which means that there exist sequences —)• 0, (u”, u"') {u, v) and tn i 0 such that 

tnw"^ = F{y -h z -h tnu"^) = y + tnU^ + H{z + tnv'^) for all n. 

Consequently, —y — tn{vF — w'^) = H{z + tnv'^), i.e., z + t„u” G S(tj + tn{u^ — w^))■ This 
shows that {u,v) £ Tgphsiv^^) or equivalently v £ DS{y\'z){u). The desired inclusion 
then follows by the arbitrariness of v in the set {u G Z: 0 G DF{y,'z){u,v)}. □ 

The following lemma characterizes a relation between the graphical derivative of the 
normal cone multifunction Mk : X ^ X and the directional derivative of 11;^. 
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Lemma 2.4 Let (^,y) G gphMx be given. Then, for any Z\,Z 2 G X, we have that 

{Zi,Z2) G ^ G ^ g 

^i\'k{x + y-,Zi + Z2) =Zi 
^ yi'ko {X + y]Zi + Z2) = Z2, 

where ,y) denotes the inner tangent cone o/gphA/x at the point {X,y). 

Proof: By the Moreau’s decomposition theorem [11], Hk^X + 3^) = X + y — IIk° {X + 
3^). This implies the third equivalence. The second equivalence is immediate by the 
relation between gphMx and gphjVi^o. Thus, it suffices to establish the first and the 
third equivalence. For this purpose, we first establish the following implication: 

{ZuZ2)£gphDMK{x\y) ^ u'K{x + y-,Zi + Z2) =z^. (12) 

Let {Zi,Z 2 ) G gph. DJ\fK{X\y) = TgphAf^iX,y)- Then there exist sequences tn i 0 and 
{Zf,Z^) {Zi,Z 2 ) such that (Tf,T) + , Z”) G gpliTV^. Recall that 

gphJ\fK = {{X,y)GKxK°: UK{x + y) = xj 

= [{x,y) eK X K°: n;^o(T' + 3;) = 3;} 

= [{x,y) eK X K°: {x,y) = oy (13) 

So, UK{X + y + tn{Zf + Z^)) =X + tnZf. Along with I].K{X + y) = A, it follows that 

^ [YIk^X + T + tn{Zf: + Zlf)) - IIk{X + T)] = Zf for all n. 

Since •) is directionally differentiable in the Hadamard sense, taking the limit n —>■ 

+00 to the both sides of the last equality yields that 11^(A + y]Zi + Zf) = Zi. This 
shows that the implication in (12) holds. Next we establish the implication that 

Yi'j,{X + y-Z,+Z2)=Z, (Zi,Z2)G7^ph^^(A,T). (14) 

To this end, let {t„} be an arbitrary sequence with t^ f 0, and for each n dehne 

[x^,y^) := {i{k{x + y + tn{Zi + Z 2 )),YiK^{x + y + tn{Zi + Z 2 )) GgpWi^. 

Clearly, {X^,y'^) {X,y). Also, by ni^(A’ + y) = X and ni^o(A + y)=y, we have 

X^-X = IiK{X + T + tn{Zi + Z2)) - IiK{X + T), 

y^ -y = YlK^{X + y + tn{Zi + Z2)) - n^o(A + T). 

Together with the directional differentiability of lix and IIx", it follows that 

lim (n'^(A + T;Zi+Z 2 ),n'^o(A + T;Zi+Z 2 )) = {Zi,Z 2 ). 

n^oo tn / \ / 
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This shows that {Zi, ^ 2 ) G T^phA/'x^'^’^)' '^gphNK^^^y) ^ %pw^K{y^y) = 

gphZ)A/A'(<T|T)- The first and the third equivalence follow from (12) and (14). □ 

By [5, Proposition 4.3] the set K is second-order regular (see [1, Dehnition 3.85] for 
the definition). Next we recall from [1, Theorem 3.45 & Theorem 3.86] the second-order 
optimality condition of problem (1) at X, where a{-,'T^{G{X),G'{X)Z)) is the support 
function of T^{G{X),G'{X)Z), the outer second-order tangent set of K at G{X) in the 
direction G'{X)Z, and C{X) denotes the critical cone of problem (1) at X with the form 

C(X) := {Z e X I h'(X)Z = 0, G'(X)Z G Tk{G(X)), {f(X),Z) = O}. (15) 

Lemma 2.5 Suppose that X = (t,X) is a locally optimal solution of problem (1) and 
Robinson’s CQ holds at X. Then the following second-order necessary condition holds: 

sup _ {{Z,Vl;^L(X-X,y)Z) - a{y,T^{G(X),G'(X)Z))} >0 yzec(x), 
{\,y)eMix) 

where for any A € and T G X, \7‘j^^L{X; X,y) is the Hessian of L{-; X,y) at X. 
Conversely, suppose that X is a stationary point of problem (1) and Robinson’s CQ holds 
at X. Then the following second-order sufficient condition 

sup _ { {Z, VlxL{X- A, y)Z) - a{y, Ti{G(X), G'(X)Z ))} > 0 VZ G C(:T)\{0}, 
(x,y)eM{x) 

is necessary and sufficient for the quadratic growth of problem (1) at X. 

By [5, Proposition 4.2] the sigma term in Lemma 2.5 is computable. To introduce its 
expression, for any given X G we assume that X admits the SVD of the form 

X = U[DmgiaiX)) 0]v'^ = U[Dmg{a{X)) O] [Fi (16) 

where U G O"* and V = [Fi V 2 ] G O” with Vi G and V 2 G , Dehne 

o:= {i |crj(X) > 0, 1 <i < m'^,b:={i \ai{X)= 0, 1 <f < m}, c:={m-|-1,..., n}. (17) 

We use z/i > z ^2 > • • • > to denote the nonzero distinct singular values of X, and write 

ai := {i I o'i(X) = vi, 1 <i < m} for Z = 1,2,..., r and a^+i := b. (18) 

Lemma 2.6 Let {(J,X), (C,r)) G gphMx be given. For any {t,H) G M x write 

Tr(hx)((C,r),(r,77)) =a((C,r),ri((LX),(r,77))). 

If X = 0, then ((C) T), (r, 77)) =0. If X ^ 0, by letting Vi > V 2 > ■ ■ ■ > Wo be the 

first ro nonzero distinct singular values of X, writing /3 := | ai{X) = ak(X) :=7'}, 
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and supposing that X = X -\-T has the SVD as in (16) with the index sets a, b, c and 
ai {I = 1,2,... ,r) given by (17)-(18), then for any (r, H) G M x we have that 


ro 


i=i 

+ (j:gg{T),2PjB{H){B(X) -VI)^B{H)P0 

if ak{X) > 0; and otherwise 

ro 

T^(l,x)((C,r), iT,H)) := -Cj2^d2Pl [B{H)iB(X)-V,I)^BiH)]r 


i=i 


+ ([S^^(r) 0],2[u'^pHX^HVfs u'^pHX^HV2]), 


where X^ denotes the Moore-Penrose pseduo-inverse of X, B : gm+n ^ linear 

mapping defined by B{Z) = ^ for Z G and P G is defined by 


P = 


Z'^ 0 

1 


V2 


Pa Ub 0 Ub ul 


Fa Vb VW2 -Vb -V. 


3 Main results 

We first establish a proposition to provide an equivalent characterization for the point 
pair in gphZlA/x°((C, r)|(t, X)). The proof of this proposition is put in Appendix B. 

Proposition 3.1 Let ((J, X), {f,T)) G gplnVic be given. Write X := {t,X) = (t,X) + 
(C, T). Let X have the SVD as in (16) with a, b, c and ai (/ = 1, 2,... , r) defined by (17) 
and (18). Then, ((At, AX), (A^, AT)) G gphL)A/V° ((C) r)|(t, X)) if and only if 

' {XC,XV)eCk{X), (19a) 

< {At, AX) - (0,C7X(I7^ArF)F^) G [Ck{X)]°, (19b) 

^ ((At, AX), (AC, AT)) = -T(j^^)((C,r),(AC,Ar)), (19c) 

where Ck{X) = 7ic((t, X))n(C,r)-’- is the critical cone associated to the complementarity 
problem K 3 (t,X) ± {t,Y) G K°, and X is a mapping with X(Ar) = 0 if X G intX or 
mtK°, and otherwise X(Ar) given by (69) ifak{X) > 0 and given by (71) if ak{X) = 0. 

Now we are in a position to establish the main results, which are stated as follows. 

Theorem 3.1 Let X = (t,X) £ ^ be a stationary point of (1) with (A,T) £ A4{X). 
Wmte y = (r,F) and G{X) = (Gi(F), G 2 {X)). Let X = Y + G 2 (X) have the SVD as 
in (16) with a, b, c and ai {I = 1,... ,r) given by (17) and (18) when X ^ int X U int K°. 


8 







(a) If the second-order sufficient condition of problem (1) holds at X and (A, y) satisfies 
the strict Robinson’s CQ, then the multifunctions J and J defined in equations (9) 
and (10) respectively are locally upper Lipschitz at the origin for 

(b) If the multifunction J is locally upper Lipschitz at the origin for {X,\,y), then 
the strict Robinson’s CQ holds at (A,y). 

Proof: (a) Since ni^(-) is directionally differentiable everywhere and globally Lipschitz 
continuous by [5], and the function / and the mappings h and G are twice continuously 
differentiable, by [1, Proposition 2.47] the mapping 'L is directionally differentiable with 

/ -A^, + A, y)l^x + [h'{y)]*Ax + [G'{X)]*Ay \ 

-As,+h'{y)Ax , (20) 

VG'(y)A;r - A^^ - U'j,{G{X) -dc + y-, G'{y)Ax - As^ + Ay)) 

where [h'{X)]* and [G'{X)\* denote the adjoint of h'{X) and G'{X), respectively. Let 

^:= {(A;r,AA,Ay) eXxRP xX I $'((0, y, A,y); (0, A;r, Aa, Ay)) = O}. 

By Lemma 2.3 and the directional differentiability of ^ and 'L, it follows that 

Zlj((0,0,0)|(:L,A,y))(0,0,0) C^ = Zlj((0,0,0)|(:L,A,y))(0,0,0). (21) 

Next we prove that ^ = {(0, 0,0)}. Clearly, (0,0, 0) S Suppose on the contradiction 
that there exists a nonzero Ajs- = [Ax, Aa, Ay) € Together with (20), we have that 

VlxL(X,X,y)^x + [h'(X)]*Ax + [G'(:T)]*Ay = 0, (22) 

h'(X)Ax = 0, (23) 

U'k{G(X) + y-G'(X)Ax + Ay) = G'(X)Ax. (24) 

Making the inner product with Ax for the two sides of (22) and using (23) yields that 

{Ax,V\x^{XXy)^x) + iG'{X)Ax,Ay) = 0. (25) 

From equation (24) and Lemma 2.4, it immediately follows that 

{Ay,G'{X)Ax) £gvhDNK^{y\G{X)). (26) 

By (19c) of Proposition 3.1, {Ay,G'{X)Ax)= —TQ^-^~^{y,G'{X)Ax)- Then, we have 

{Ax,VlxL{XXy)^x) - yG{x){y^G'{X)Ax) = 0. (27) 

In addition, from (26) and (19a) of Proposition 3.1, G'{X)Ax G Tk{G{X)) ny*~, which 
along with (22) and L'p^{X]\^y) = 0 implies that Ax G C{X). So, by (27) and the 
second-order sufficient condition, we have Ax = 0. Substituting A;r = 0 into (22) 
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yields that [h'{X)]*/S.\ + [G'{X)]*lly = 0. Since A;t’= Oj equations (26) and (19b) imply 




that Ay G [TK{G{X))f^y ]°. Then, (AA,Ay) G 


/i'(^) 

G'(X) 


n 


lTK{G{x))ny^\ 


Thus, the strict Robinson’s CQ at (A,T) implies that (A^, Ay) = (0,0). Consequently, 
{A;y, Ax, Ay) = 0. This yields a contradiction. Hence, Z1^7((0,0,0)| (T”, A, T)) (0,0, 0) = 
^ = {(0,0,0)}, andjhen {(0,0,0)} C T)j((0,0, 0)|(:T, A,T)) (0,0, 0) C {(0,0,0)}. By 
Lemma 2.2, J' and J' are both locally upper Lipschitz at the origin for {X,X,y). 

(b) Suppose that the strict Robinson’s CQ does not hold at (A,T)- From (6), we have 


/i'(^) 

G'(X) 


+ 


0 


TKiG{x))ny^^ 

which is equivalent to saying that there exists 0 7^ (A,T) € 




X X such that 


(28) 


(A,T)G 


/i'(^) 

G'(X) 


n 


[TKiGix))ny" 


Consequently, we have that [h'{X)]*X + [G'{X)]*y = 0 and the following relation holds: 

T G {TK{G{X))r^yy = c\{Mk{G(X)) + m) = (29) 

where the last equality is due to [1, Example 2.62]. By the definition of contingent cone, 
it is easy to verify that if Z G Ty-^(g(^))(T), then (0, Z) G TgphAfK{G{X),y). Thus, 

equation (29) means that (0,T) G TgphAfKiG{X),y). Using Lemma 2.4, we have that 
Ii'K{G(X)+ y-y) = 0. Together with [h'(X)]*X + [G'(X)]*y = 0, we obtain that 

T'((:T,A,T);(0,A,T)) =0. 

By Lemma 2.3, we have (0, A, T) G DJ ((0, 0,0)|(T1, A, T))(0,0, 0) = {(0, 0,0)}, where the 
equality is due to the locally upper Lipschitz J at the origin for (X,X,y) and Lemma 
2.2. Thus, we get (A,T) = 0, a contradiction to (A,T) 7^ 0. The proof is completed. □ 


Remark 3.1 (a) From the inclusion relation in (21) and Lemma 2.2, it is not hard to 
see that the locally upper Lipschitz of J at the origin for {X, A, y) implies that of J at 
the origin for {X,X,y). Hence, when the condition of part (b) is replaced by the locally 
upper Lipschitz of J at the origin for (X,X,y), the strict Robinson’s CQ may not hold. 

(b) Let Ll be the KKT point set of (1). By Lemma 2.1, the locally upper Lipschitz of ff 
at 0 for yV = {X, A, T) means that there are a constant 1? > 0 and a small e > 0 such that 

dist((T',A,T),ll) < ||(T',A,T) -((T,A,T)|| <r?||T(T',A,T)|| V(T', A,T) G ]B(W,e), 

which provides a local error bound for estimating the distance from any points in the 
neighborhood ofW to fl, and the bound is only related to the KKT residual ||T(T’, A, T)|| • 
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(c) By the definitions of ^ and the multifunction J can be equivalently written as 
J{6) = {{X,X,y)eXxRPxX \ ^(<^,A,3^) GS + MxxRpxK^{y,X,y)}- (30) 

Thus, by equation (30) and Lemma 2.1, the locally upper Lipschitz of ff at the origin for 

W = {y, A, 3^) implies that there exist a constant i? > 0 and a small e > 0 such that 

dist((Af,A,3^),fi) < 1?(||^'(A:’,A,3^)|| + ||ni^(3^)||) V(A:’,A,3^) G B(W,e). 
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Appendix A 

Lemma 1 Let {y,z) £ gph5 for a multifunction 5 : Y ^ Z. Then, the following two 
statements are equivalent: 

(a) there exist a constant /i > 0 and neighborhoods U of y and V of z such that 

V n 5(y) = {z} and 5(?/) H V C {z} -\- fj,\\y — yjjlBY for all y 

(b) there exist a constant y' >0 and a neighborhood V of z such that 

Vn5(y) = {z} and S{y) DV C {z}-\-fj,'\\y— yWMY for all y £ Y. 

Proof: Clearly, (b) implies (a). It suffices to argue that (a) implies (b). For this purpose, 
we assume that (a) holds for neighborhoods U = ]B(y, S) and V = Mifz, e) with (5 > 0 and 
e > 0. We proceed the arguments by the two cases as shown below. 

Case 1: e > 5. We show that (b) holds for V' = M{z,e') with e' = |. Since e' < e, it is 
clear that V' riS{y) = {z}. Moreover, since part (a) holds, it immediately follows that 

5(2/) nB(z,e') C {z} + y,\\y -y\\MY when y £ lB(y,5). 

In addition, by noting that M{z,e') C {z} + |(5By, we also have 

5(2/) nB(z,e') C B(z,e') C {z} + \\y - ypY when y ^ B(y, 5) 

Thus, part (b) directly follows from the last two equations with y,' = max(/i, 1). 

Case 2: e < 6. Using the same arguments as for Case 1 can verify that (b) holds for 
V' = B(z, e') with e' = e/2. Here, we omit the details for simplicity. □ 
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Appendix B 

This part includes some lemmas used for the proof of Proposition 3.1 and the proof of 
Proposition 3.1, which requires the mappings Q: and T-L: —>■ as 


g(Z) := (Z + Z^)/2 and n{Z) := (Z - Z'^)/2 for Z e (31) 

and the matrices Ti, ^2 and J-'E ^mx(n-m) associated to the given A, A 


T)., - 1 


if (Ti(A) / adA), , . . i 

Ti{x)-aj(x) ^ !■> for j g { 1 ,...,m}, 

0 otherwise, 


ix)laf(x) if c^*(A)+aj(A) / 0 , for i,j e 
0 otherwise. 


u A) fffor i g { 1 ,..., m},j g { 1 ,..., n—m}. 
0 otherwise. 


(32) 

(33) 

(34) 


Unless otherwise stated, in the sequel, when A has the SVD as in (16), we write 
Z = u'^ZV and Zi = u'^ZVi for any Z g 


Firstly, we recall from [5, Lemma 3.15] a result on the projection of {t,X) onto K. 
Lemma 2 Let {t,X) ^ int AUint A° be given. Write (t,X) = n;^(t. A) and a = (7(A). 

(i) Ifo'k > 0, then there exist 9 > 0 and u g such that a = (7(A) — 6u with Ui = 1 

fori = l,...,ko, 1 > Uko+i > ■■■>Uki >0 with YliLko+i Ui = k - ko, and Ui = 0 
for i = ki + l,... ,m, where ko g [0, k — 1] and ki g [k, m] are integers such that 

(7l ^ ••• ^ (7fcQ> — ••• — <Jk — ■■■ — (^ki •— ^ ^ (7fci+l^ ■■■ ^ <7m^ 0. (35) 
In the subsequent discussion, for this case we always write 


a:= ,ko}, /3:={ko + l,.. .,ki}, 'y:={ki + l, ... ,m },7 = {1,... ,m}\ 7 . (36) 


(a) If (Jk = 0, then there exist 0 > 0 and u g such that a = (7(A) — 6u with Ui = 1 
for i = 1,... ,ko and 1 > Uko+i > Uko +2 > ■■■>Um>0 with Yl'lLko+i < k - ko, 
where 0 < ko < k — 1 is an integer such that 

(7l ^ . . . ^ (7fcQ > (JkQ-\-l — ... — (7/j — ... — (7m — 0. (37) 

In the subsequent discussion, for this case we always write 


a\= {1,2, ... ,ko} and I3\= {ko + l,ko-\-2, ... ,ni}. (38) 


Also, for the two cases we sometimes use the partition for /3 := /3i U /32 U /33 with 
(di := {i e /3 \ Ui = 1}, /32 := {f G /3 I Ui g (0,1)} and ^3 := {i g /? | Uj = 0}. 
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Next we recall from [5, Proposition 3.16] the expression of the directional derivative 
of the projection operator Hk at {t,X) ^ int iiT U mtK°, which is stated as follows. 

Lemma 3 Let {t,X) ^ int iPUint iP” be given. Write (t,X) = and a = (^{X). 

Suppose that X has the SVD as in (16) with a, b, c and ai (I = 1,2,... ,r) given by (17) 
and (18). Then, the directional derivative oJUk {t,X) along {t,H) G M x is 

n),((t,X);(T,F)) = {^o{t,D{H)),U~{t,D{H))v'") 


where ^q{t,D{H)) and 'E[t,D{H)) will be stated by ak > 0 and ak = 0, respectively. 

Case 1: > 0. Let ro,ri G r} be such that a = U[=i®Zi/5 = U[iro+i 

7 = US+i ai, where a,(3,j and 7 are defined by (36) with integers ko G [0, fc —1] and 
ki G [k,m] such that equation (35) holds. Write W := M x x x • • • x For 

any {(, W) G W with W = [Wi W 2 ■ ■ ■ Wrfi G x x • • • x we define 

<ho(C,M/) :=MC,f^{W)) G M and <l>i{C,W) :=RiDmg{MC, i^{W)))RJ G §1“*' 

withRi G fori = l,...,ri, where {4>o{C,K{W)),(t)i{C,K{W)),... ,(t)ri{C,K{W))) 

with k{W) := (A(VPi), A(iy 2 )) ■ ■ ■ j ^(^ri)) G is the unique solution of the problem 

min \[{V - Cf + \\d - i^{W)f] 
s.t. {ea,da) + S(^k-ko){dp) < g 

if {t,X) G hdK, and otherwise is the unique optimal solution of the problem 

min ^[iv - Cf + \\d - K{W)f] 
s.t. {ea,da) + si^k-ko){dg) <v 
{ea,da) + {ug,di3) = r]. 


Here, . 


_ ^^k—kQ t 


is defined by S(k-ko)(^) ■= Ei=i°4 M z G 


E(t,D(H)):=T(H) + 


$i(t, H(77)) 0 0 0 

0 0^0 
0 0 ^ri{T, D{H)) 0 

0 0 0 


Then, 

0 

0 

0 

Hbc A 


(39) 


where D{Z) and T{Z) for any Z = [Zi Z 2 ] G with Z\ G are defined by 

D{Z) := {G{Za,a,),Q{Za,a,)....MZar,a.,)) (40) 

and 


T{Z) \Ei o Q[Zi )-\-£2 ° T~L{Zi) 


(41) 
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Here “o” means the Hardmard produet. In partieular, from [5, Page 127], we know that 


nci (r, D{H)) = (c^o(t, D{H)), $1 (r, D{H)), (r, D{H))), (42) 

where Ci C W is a closed convex eone which, if {t,X) G hdK, has the form 

Cl = {(C, G w I EliMWi) + S(k-k0){'^h{w)) < c}, 

and otherwise takes the following form 


ro ro 

Cl = {(C,H^)gW |^tr(H^0+S(fc-A:o)('^/3W)< C,J2^^{Wi)+{u^,K^iW)) = c]. (43) 


1=1 


1=1 


Case 2: = 0. Let rg G {1,2,... ,r} be sueh that a = U[=i 

where a and /3 are defined by (38) with integer kg G [0,k — 1] sueh that (37) holds. Let 
W := M X Sl“il X • • • X §1“’-! x MlMx(|t-|+|c|). por any (C, W) with W = [Wi ■■■ Wr Wr+i] 

G Sl“il X • • • X §!“’■! X mI^|x(I^I+I'^I), we define 4>o(C, W) := MC, '^(^)) ^ ^/(C, W) := 

RiBmg{fii{C,K{W)))Rj G Sl“*l with Ri G Ol“'l(Tyz) for I = 1,2,... ,r, and^r+i{C,W) := 
U[Dmgl4>r+i{C,i^{W))) 0]!/^ G Ml^lxd^l+I'^l) with {U,V) G Ol^l’(l^l+l"l)(W^+i), where 

(MC, >^{W)), MC, ..., (fr+iiC, k{W))) with k{W) = (A(14^i),..., \{Wr),a{Wr+i)) 

is the unique optimal solution of the following convex minimization problem 

min \[{V - Cf + \\d - K{W)f] 

S.t. {ea,da) + M/3||(fc-fco) < V 

if {t, X) G hdK, and otherwise is the unique optimal solution of the eonvex problem 

min i [{r] - (f + \\d - n{W)f] 

s.t. {ea,da) + \\dfl\\(^k-ko) ^ h 
{ea,da) + {Ul3,dj3) = T]. 


Hero, II • II . 


E{t,D{H)):=T{H) + 


is defined by \\z\\(k-ko) ■= Yl!i=i°\A\ for z G RI^L Then, 
4)i (r, £>(77)) 0 0 0 


0 0 ^ 0 
0 0 ^r{T,D{H)) 0 

0 0 0 4>r+i(r,Z)(F)), 


(44) 


where D{Z) and T{Z) for any Z = \Z\ Z<i\ G M™xn ^ -gmxm given by 

D{Z) := {G{Za.,^ai),G{Za2a2), ■ ■ ■ ■,G{Zarar)A^bb -^bc]) (45) 

and 


■= [^1 o Q{^l) + C 2 o H{Zi) iFaUl3,c ° Zaul3,c] ■ 


(46) 
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Here “o” means the Hardmard produet. In partieular, from [5, Page 129], we know that 
Uc,{t,D{H)) = (47) 

where C 2 is a elosed convex cone which, if {t,X) E hdK, has the form 

ro 

C 2 := {{C,W) G W I J2tT{Wi) + ||AC/3(Vr)||(fc-fco) < C}, 

1=1 

and otherwise takes the following form 


ro ro 

C2.= {iC,W)GW \J2^T{Wi)+\\n^iW)\\^k-ko) < C,Y.^TiWi)+{u0,K0{W)) = c}. (48) 


1=1 


1=1 


The following lemma provides a crucial result on the directional derivative of Hk, 
which can also be found from the proof of [5, Proposition 4.5]. 

Lemma 4 Let {t,X) ^ int ii'Uint ii'” be given. Write (t,X) = IlK{t,X) and a = o'{X). 
Let X have the SVD as in (16) with a, b, c and ai (/ = 1, 2,... , r) given by (17) and (18). 


(i) If ak > 0, then ((t, X)] {At, AX) + (A^, AP)) = {Af, AP) if and only if 


' AC=4>o(r,L>(AX + Af)); (49a) 

g{Afa,ai) = HpD{AX + Af)), l = l,2,...,rr, (49b) 

[Q{AXi)]aiat, = 0, I I'and 1,1'= 1,2,... ,ro; (49c) 

, [0(Afi)]a(a^, = 0, I I'and 1,1'= ro + 1,... ,ri; (49d) 

and 


[g{Af^)u - {£i)^^ o [g{Aw)u = {£i)^fi o [g{AXi)u- 
[g{Af,)u - o [g{Af,)u = {£i)^^ o [g{Ax,)UY, 

[g{Af^)]p, - (Ti)^^ o [g(Af i)]^^ = (Ti)^^ o [g{AX,)]^,-, 


'H{ATi)-£ 2 on{ATi)=£ 2 on{AXi) with £2 

AA^c] — 0. 




(50a) 

(50b) 

(50c) 


0 


;(50d) 


(50e) 

(50f) 
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(n) If Ok = 0, then ((t, X)] {At, AX) + (A^, AF)) = (A<^, AF) if and only if 


' AC=$o(T,-D(AA + Ar)); (51a) 

[AFfefe AFfeJ = ^r+i (r, D{AX + AF)); (51b) 

Q{Afaiai) = ^i{t,D{AX + AT)), l = l,2,...,r- (51c) 

< [^(AXi)]a;a^, = 0, Z //'and/,/'= 1,2,... ,ro; (51d) 

AFaja;, = 0, a.ndl,l'= ro + l,...,r+T, (51e) 

[g(Ari)],^ - {£,)^^ o [^(Af i)]„^ = o [g{AX,)U 0 ; (51f) 

,Ara,c = 0, l = ro + l,...,r (51g) 

and 

' n{Af^^) - o ^(Ar„„) = o n{Ax^^y, ( 52 a) 

< [^(Afi)]„^ - (f2)„^ o [niAf^)U = o [niAX^)U; (52b) 

^Afac-J^ac°Afac = XacoAXac- (52c) 


Proof: By Lemma 3, n)^((i, A); (At, AA) + (A(',AF)) = (A^,AF) if and only if 
AC =^o{t,D{H)) and AT = E{t,D{H)) with t = At+AC, H = AX + AT, (53) 


where E{t, D{H)) is given by equation (39) if ak > 0, and otherwise is given by (44). 

(i) It suffices to argue that the second equality of (53) holds with E{t,D{H)) given by 
(39) if and only if (49b)-(50f) hold. Suppose that the second equality of (53) holds with 
E{t, D{H)) given by (39). We immediately have (50e). Notice that (fi)^^ = 0. By the 
expressions of (^ 2)77 and Aye, we obtain (50f). By the symmetry of £i and 4>;(r, D{H)), 


'H{ATi)=£2o'H{AXi + ATi). 

This implies (50d). By the symmetry of £2 and 4>;(r, D(iF)), we have that 


g{ATi)=£iog{AHi) + 


'^i{t,D{H)) 

0 

0 

0 


0 0 0 

0 ^ 0 

0 ^r,{T,D{H)) 0 

0 0 g{H^^)_ 


(54) 


(55) 


By this, we obtain equations (49b)-(50c). By using a = a — 9u and (35), we have that 

' = Eaiai,, for I / l' and 1,1' ^{l ,... ,ro} or l,l'^{ri + l ,... ,r-hl}; 

< [£i]aiai,=0, for / / F and Z,Fe{ro + 1 ,... ,ri}; 

, [^i]a,ai = 0 , for / = 1 , 2 ,... ,r-hl. 
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Together with (55), we obtain that (49b)-(49d) hold. Consequently, the necessity follows. 
Conversely, suppose that (49b)-(49d) and (50a)-(50d) hold. Then (54) and (55) must 
hold. Along with (50e)-(50f), we obtain the second equality of (53). 

(ii) It suffices to argue that the second equality of (53) holds with D{H)) given by 
(44) if and only if equations (51b)-(52c) hold. Suppose that the second equality of (53) 
holds with 'E{t, D{H)) given by (44). Firstly, we readily get equation (52c). By noting 
that {£i)bh = 0, {£2)bb = 0 and = 0, equation (51b) and (51g) also hold. Notice that 


and 


niATi) =£2 0'HiAHi) + 


^aU{/3\b),aU{/3\b) 

^b,aU(/9\b) 


^aU(/3\b),b^ 

■H([c!>r+i(T,n(ff))h) 


(57) 


g(Ari) = £iog(ffi) + 


■4>i(r,T>(F)) 

0 

0 

0 


0 0 0 

•. 0 ^ 0 

0 ^r(T,-D(R)) 0 

0 0 g([^r+i(T,n(R))h)_ 


(58) 


due to the symmetry of £ 2 , £1 and (r, for / = 1 ,..., r, where [4>r-i-i {t, DiH))], 

is the matrix consisting of the first m columns of ^r+i{T, . Equation (57) implies 

that (52a)-(52b) hold, while (58) implies that (51d) holds. By a = a — 6u and (37), 


' [Si]aiai, = Eaiai,-, for I / l' and /,/'€{!, 2 ,... ,ro}; 

< [£i]aiat,=0, for / //'and Z,/'E{ro + l,... ,r + l}; 

= 0 , for Z = 1 , 2 ,... ,r + l. 

Together with (58), we get equations (51c)-(51f). Consequently, the necessity follows. 
Conversely, suppose that (51c)-(51f) and (52a)-(52b) hold. Then (57) and (58) must 
hold. Together with (51b), (51g) and (52c), we obtain the second equality of (53). □ 

Next we provide an equivalent characterization for the critical cone of K at {t,X) -\- 
(C,r) associated with the complementarity problem K 3 {t,X) T (t, T) G K°. 

Lemma 5 Let {(t, X), {(,,£)) G gpluA/)^ be given with (t-bCi-^ + r) ^ intiF UintiF°. 
We write {t,X) = (t,X) + (CjB), a = cr{X) and a = cr(X). Let X have the SVD as in 

(16) with the index sets a, b,c and ai (/= 1, 2,... , r) defined by (17) and (18). 

(i) If Ok > 0, then (r, Z) G Tk(t, X) C (C, T)-*- if and only if (r, D{Z)) G Ci. 

(ii) If Ok = 0, then (r, Z) G Txff, X) C (C, T)-*- if and only if (r, D{Z)) G € 2 - 

Proof: (i) Let ro,ri G {l,2,...,r} be such that a = P — U[iro+i 

7 = U[ir\+i where a, /3 ,7 and 7 are defined by (36) with G [0, A:—1] and ki G [k, m] 
such that (35) holds. Also, hy o = o — 9u (see Case (i) of Lemma 3) and (C, T) G bd iF°, 

C = t-t = -e and r = X -X = U[Diag{9u) 0]F^. (60) 
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Let {t, Z) be an arbitrary point from TK{t^X) n From {t, Z) S TK{tiX) and 

the expression of TK{t,X) (see [5, Equation(4.1)]), it follows that 

E£itr(ifI,zF„,) + Etf'’F{e(z^^)) < T, 

while from (t, Z) G ((^jF)-*- and equation (60), it follows that 

0 = TC + (Z,r} = -ffT + 0EZMK^Va,) + 0(Z^^,Dmg(u^)} 

= -Ot + eti{ulzVo.) + e{G{Zp0),Bmg{up)) 

< -Ot + eti{ulzVo.) + eu}x{g{Zp0)) 

< -Or + etriulzv^) + eY!lZi°h{Q{Zpf,)) 


where the first inequality is using the von Neumann’s trace inequality, and the last one 
is due toui 3 = v}p and Ui = k — ko- So, (r, Z) G Txit, X) n (C, F)-*- if and only if 

(a(^/3/3), Diag(M/3)) = u]i\{g(Zfip)) (61) 


and 

iriZaa) + < T and tr(Z„Q,) + {u^, X{g(Zi 3 ii))) = r. (62) 

By the von Neumann’s trace inequality, equation (61) is equivalent to saying that g{Zpis) 
and Diag(’u^) have a simultaneous ordered eigenvalue decomposition, which by the proof 
of Case (i) of [6, Proposition 5.1] is equivalent to saying that 


0(^00) 


g{Zp^is^) 0 0 

0 y 

0 0 g{H^^0^)_ 


where /3i, (32 and /Ss are the index sets defined as in Lemma 2. Together with (62), the 
definitions of the operator D in (40) and the set Ci in (43), we get the desired equivalence. 

(ii) Let ro G {1,... , r} be such that a = U[=i /3 = IJ[ir^o+i where a and /? are 

defined by (38) with k^ G [0, fc—1] such that (37) holds. Now j3\ = Org+i, = b and 
/32 = U =ro +2 Also, hy a = a — 9u (see Case (ii) of Lemma 3) and (C, F) G bdiL°, 

C = t-t = -e and T = X -X = U[Diag{9u) 0]F^. (63) 


Let {t,Z) be an arbitrary point from TK{t,X) Cl (^jF)-*-. From {t, Z) G TK{t,X) and 
the expression of TK(t,X) (see [5, Equation(4.2)]), it follows that 

E?=MulzVa,) + E^Zl°<^^{[Z00 Zpy<r, 


while from {t,Z) G (C,r)-’- and equation (63), it follows that 

0 = rC + {Z,T) = -9t + 9Y:ilMul^ZVa,) + 9{[Zpfi [Diag(ll^) 0^,]) 

< -9t + 9ii{ulzvy + 9u}a{[Z0p Z^^j) 

< -Ot + 9iT{ulzvy + ^/3c]) 
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where the first inequality is using the von Neumann’s trace inequality, and the last one 
is due to and X^ie/3 Ui = k — ko- So, (r, Z) G 7A'(t, X) n (C, T)-*- if and only if 

{[Z/sp Z,gc], [Diag(u/3) {)pc]) = u]ia{[Zpp Zp^]) (64) 


and 


tr(Z„«) + Y!i=i°^i{[Zpp Zpc]) < r and tr(Z„„) + {up,a{[Zpp Zp^])) = r. (65) 

By the von Neumann’s trace inequality, equation (64) is equivalent to saying that \Zpp Zpc\ 
and [Diag(lt/3) Qpc\ have a simultaneous ordered SVD, which by the proof of Case (ii) of 
[6, Proposition 5.1] is equivalent to saying that [Zpp Zp^ has the following structure 


[Zpp Zpc] 


y 

^rQ +1 +1 

0 

0 

0 


0 

^rQ +2^rQ +2 

0 

0 


0 0 0 

0 0 0 

Zarar J)* 

0 ^bc 


( 66 ) 


with Zaiai G for I = ro + 1,..., r. Together with equation (65), the definitions of the 
operator D in (45) and the set C 2 in (48), we obtain the desired equivalence. □ 

Now we give a characterization for the negative polar cone of the critical cone of K at 
(t,X) + (CjT) associated with the complementarity problem K 9 {t,X) T {t,Y) G K°. 

Lemma 6 Let {(t, X), {(^,T)) G gplrA/)^ be given with (t + ^, X + T) ^ miK Uintif"°. 
We write {t,X) = (t,X) + (Cjh), a = cr(X) and a = cr(X). Let X have the SVD as in 
(16) with the index sets a,b,c and ai (Z = 1, 2,... , r) defined by (17) and (18). Suppose 
that ((At, AX), (A(^, AT)) G gphUA/'A'o (((^, r)|(t, X)). The following statements hold. 

(i) IfWk > 0, then (At, AX) - (0,I7X(Af )F^) G [71^(t,X) n (C,r)^]° if and only if 

r^t + Y:ZliS{XXa,a,),G{Za,a,)) <0 V(t, Z) G Tk(C X) C (C, T)^, (67) 


where X: —)• jg a mapping defined by (69). 

(ii) IfWk = 0, then (At, AX) - (0,I7X(Af )F^) G [T^(t,X) n (C,r)^]° if and only if 

rAt + ^[^i(0(AX(jjaJ, ^(XqjqJ) + (AXbb, Xbb) + (AXbc, .^bc) ^0 (68) 

for any {t,Z) G TK{i,X) C (C,r)-L, where X: ^ is defined by (71). 

Proof: Since ((At, AX), (A^, AT)) G gphL^A/i^o ((C, r)| (t, X)), by Lemma 2.4 we have 
that n)^((t, X); (At, AX) + (A(',Ar)) = (A^,Ar). So, the results of Lemma 4 hold. 
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(i) Let ro, ri G {1, 2,... , r} be such that a = |J[=i /3 = U[iro+i = U[i\+i 

where a,l3,'y and 7 are defined by (36) with the integers G [0, fc —1] and ki G [k,m] 
such that (35) holds. For any (r, Z) G X, from equation (50f), we have that 

rAt + (AX, Z) = rAt + (AX, Z) = rAt + (AXi, Zi) + (AX2, Z2) 

= rAt+(g(AXi),g(Zi)) +(Lt(AXi),Lt(Zi)) + (AX„„Z„,) + (AX^„Z^,) 

= rAt + ^(g(AX„,„J,g(Z„,,J) + (Lt(AXi),Lt(Zi)) + (AX«„Z„e) + (AX^„Z^e) 

«=i 

+ 2[([e(AXi)]„^, [e(Zi)]„;3) + ([a(AXi)]„^, [6;(Zi)]„^) + ([g(AXi)]^^, [6;(Zi)]^^)] 

where the third equality is using (^(AXi),Lt(Zi)) = 0. By (50d) it follows that 

([Lt(AXi)]„,„,, [llt(Zi)]„,,,) = ((02)a,a, o [Lt(Afi)]„,,,, [Lt(Zi)]„,,,) 

for I = 1,... ,ri and Z' = 1,2,... ,r + l, or I = ri + 1,... ,r + l and I' = 1,... ,ri, where 
(02)aiaj/ = [Eaiai! ~ (^2)0(0;/] 0 {£ 2 )aiaii with “0” denoting the division of entries. While 
from equations (50a)-(50c) and equation (50e) it immediately follows that 

' {[G{AX,)Ua,AG{Zi)Ua,) = ((0l)a,a,^O [g( AFi)],,., , [^(Zi)],,.,), 

< {AXac, Z^c) = {[{E - Eac) 0 Eac] ° AFac, Zac), 

, (AX^C, Xy3c) = ([(LZ - Efic) 0 X^c] O AF^C! Z^c)- 

for 1,1' = 1,2,... ,r + l and I / 1', where {Qi)aiai, = [Xa^a,, - (^Oa^a,,] 0 i£i)aiai,- From 
the last two groups of equalities, an elementary calculation yields that 

(Lt(AXi),Lt(Zi)) + {AXac, Zac) + (AX^„Z^,) +2([e(AXi)]„^, [G{Zi)]afi) 

+ 2[([g(AXi)]„^, [a(Zi)]„^) + ([g(AXi)]^^, [e(Zi)]^^)] 

= (x(Af),z) = (Fx(Ar)F^,z) 


where X: j^mxn jg ^ mapping with X(X) for any A G defined by 


X(X) := 


'^aa 

(0i)/3q O [G{Ai)\0a 

L(0i) 7a ° [Q{^l)\'ya 


+ 


(02)ao O T-LiAaa) 
(02)/3o O [^(^l)]/3o 

(02)70 O [ 0 (^l)] 7 Q 


(0l)o/3 O [^(^l)]o/3 (0 i)q 7 O [^(^l)]o 7 EacoAac 

^13/3 (® 1 )/37 o [^(^1 )]/37 Efjc^A^c 


( 01 ) 7 ^ o [G{£^l)\'yf3 O 77 O 7 C 

(02)q/3 O \H{Ai)\a^ ( 02)07 ° [^(^l)]o 7 Oac 

{&2)l3j3 oE-iAfS/s) {@2)l3'y ° [E.{M)]l3'y ^I3c 
( 02 ) 7/3 o [Q{-^l)'\')f3 O 77 O 7 C 


(69) 


with Xyc = (E^c ~ E^c) 0 Xyc- Thus, for any (r, Z) G X, it always holds that 

{{At, AX), (r, Z)) - ((0,I7X(Af )F''), (r, Z)) = rAt + X:[ii(e(AX,,„,), ^(Z,,^,)). 


In view of this, the desired conclusion of this case then follows. 
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(ii) Let ro G {1,... , r} be such that a = o; and /3 = IJ[i^o+i where a and (5 are 
defined by (38) with /cq G [0, A:—1] such that (37) holds. For any {t, Z) G X, by (51d) 

rAt + (AX, Z) = rAt + (AX, Z) = rAt + (AXi, Zi) + (AXa, Za) 

= rAt+(g(AXi),g(Zi)) +(7t(AXi),7t(Zi)) + (AX„„ Z„,) + (AX^„Z^,) 

ro 

= rAt + ''^^{G{AXaiai),G{Zaiai)) + + {^Xaci Zac) + X^c) 

«=1 

+ 2([g(AXi)]„^, [e(Zi)]„^) + ([7t(AXi)]««, [7t(Zi)]«„)+2([7t(AXi)]„^, [7t(Zi)]„^). 


From equation (51f) and equations (52a)-(52c), it is immediate to obtain that 




{[g{AX^)]a,a,AG{Z^)Ua,) 

m^x^)Ua,Anzi)Ua, 


= ((0l)a,a, O [g(Ar^i)],,,,, [g(Zi)],,,,), 

) = on{ATa,a,)AnZl)Ua,), 


{^Zaci Zac) — ac ® Za<Ai 


(70a) 

(70b) 


where (0i)aia,, = (-^aia,, - {Si)aiai,)&{£i)aiai, for / = 1,2,... ,ro and I' = ro+1,... ,r+l, 
(02)aia;, = {Ea^a^, “ (^^2)aia;,)0 (^2)aia,, for 1,1' = l,...,ro, OT I = and I' = 

ro + 1,..., r + 1, and Xac = (Eac — Eac) 0 Eac- Let X: j^mxn |^g defined by 


X(X) := 


+ 


Octa (0 i)ck/3 ^ [^(^i)]q!/3 Eac ^ -^o 

(0l)/3a o [^(^i)]/3q 0/3/3 0/3c 

{&2)aa°'H{Aaa) ( 02 )a /3 O [ 7 t(Xi)]Q ,/3 Oac 

_(02)/3o o [E.{Ai)]j3a 0/3/3 0/3c 


\/A G 


(71) 


Then, after an elementary calculation, it is easy to calculate that 
2([g(AXi)]„/3, [G{Zi)U) + ([7t(AXi)]«„, [n{ZA]aa) 

+2([7t(AXi)]«/3,[7t(Zi)]„/3) + {AXacZac) = (X(Ar),Z) = (Z7X(Af)F'',Z). 


Thus, for any (r, Z) G X, it always holds that 

((At, AX), (r, Z)) - ((0,!7X(Af )F^), (r, Z)) 

= rAt + E[°i(e(AX„,„,),g(Z„,„J) + (AX/3/3,Z/3/3) + (AX/3e, Z/3e). 


By the proof of Case (ii) of Lemma 5, (r, Z) G 7F(L E) n {C, F)-*- if and only if [Z/ 3/3 Z/ 3 c] 
has the structure in ( 66 ). For such {t,Z), we have that {AXjSc, Zj^A = {AX^c, Z^c) and 

(AX/ 3 / 3 , Z/ 3 / 3 ) = ELro+liGiEXa,aA,G{Za,aA) + {Xbt,Ztb) 

The desired result then follows from the last two equalities. □ 


The proof of Proposition 3.1. Notice that (t,X) = nx(t,X), (CT) = ^K°{t,X) 
and ((t,X),(C,r)) = 0. Let ((At, AX), (AC, AF)) G gphZlXKo(^C,_r)l(L^))- By the 
third equivalence of Lemma 2.4, ((At, AX), (AC, AF)) G Tk° xi^((C,r),(t,X)) and 

n)^((t,X);(At,AX) + (AC,Ar)) = (AC,Ar). (72) 
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Then (A(^,Ar) S TK{t,X). Thus, to prove that (A^,Ar) £ Ck{X), we only need to 
show that (A(^, AT) £ {(^, T)-*-. We proceed the arguments by three cases as shown below. 

Case 1: {t,X) £ miK. Now (CjT) = (0,0) which implies equation (19a), and Hk is 
directionally differentiable at {t,X) with X)] (t, H)) = {t,H) for {t,H) £ M x 

l^mxn^ The latter means that n'^((t. A); (At, AX) + (AC, AT)) = (At +AC, AX +AT). 
Together with (72), we have {At, AX) = (0,0) £ [Ck{X)]°, and then (19b) follows. 
Equation (19c) directly follows from (C,r) = (0,0) and (At, AX) = (0,0). 

Case 2: (t, X) £ intX°. Now (t, X) = (0,0) and (Cjh) = (t,X) £ intX°. Then 
TK{i,X) n (C,r)^ = xn (C,r)^ = {(0,0)}. So, (At, AX) £ X = [Cx(X)]°, and (19b) 
follows. Notice that Hk is directionally differentiable at (t, X) with n^((t, X); (r, H)) = 
(0, 0) for any (r, H) £ M x which means that n)^((t, X); (At, AX) + (AC, AT)) = 

(0,0). Together with (72), we obtain (AC, AT) = (0,0) £ Ck{X), and (19a) follows. 
Since (t, X) = (0,0), (19c) directly follows by (AC, AT) = (0,0) and Lemma 2.6. 

Case 3: (t,X) ^ intX U intX°. Let X have the SVD as in (16) with the index sets 
a, b, c and ai {I = 1,2,... ,r) defined by (17) and (18). Write a = cr(X) and a = cr(X). 
Let r = At + AC and H = AX TAT. We proceed the arguments by two subcases. 

Subcase 3.1: > 0 . Let ro, ri £ { 1 , 2 ,..., r} be such that a = U[=i /3 = U[iro+i 

and 7 = where a,/3,'y and 7 are defined by (36) with integers fcg £ [0, fc—1] 

and ki £ [k, m] such that (35) holds. Also, equation (60) still holds. In addition, by (72) 
and Case (i) of Lemma 4, equations (49a)-(50f) hold for this case. 

We first prove that equation (19a) holds. By equations (49a)-(49b), the definition of 
the operator D in (40), and equation (42) with Ci given by (43), we have that 

(AC,T>(Ar)) = {^o{r,D{H)),<^i{T,D{H)),...,<l>r,{r,D{H))) =Uc,iT,D{H)). (73) 

By Lemma 5, (AC, AT) £ Ck{X), i.e., (19a) follows. By the Moreau decomposition [11], 

{At +AC, D{AX+Af)) = Bci {At +AC, D{AX+Af)) + {At + AC, D{AX + Af)) 

= (AC, T>(Ar)) + Bq {At + AC, D{AX + AT)), 

which implies that {At, D{AX)) = Bco(At -|- A(^, D{AX + AT)). Notice that 

r^t + j:?=i{G{^Xa,a,)MZa,a,)) = {{At, D{AX)), {t, D{Z))) < 0 

for any (r, Z) £ 'TK(t, X) H (C, T)-*-, where the inequality is due to Lemma 5. By Case (i) 
of Lemma 6 , {At, AX) — (0,17X(Ar)B''^) £ [Cii'(X)]°. Thus, equation (19b) holds. 

We next prove that (19c) holds. By (At, D{AX)) = Bco(At -|- AC, D{AX + AT)), 

AtAC + E[ii(^(AX,^a,),a^Ar,,,J) = ((At,T>(AX)),(AC,T>(Af))) 

= (Bco(At +AC,i:>(AX +Ar)),Bci(At + AC,T>(AX +AT))) =0. 
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Together with the arguments for Case (i) of Lemma 6, we have that 

((At, AX), (AC, AT)) = (X(Ar),Ar) 

= (i^(AXi),^(Afi)) + (AX^e,Af^,) + 2([C;(AXi)]„^,[a(Ari)]„;3) 

+ 2[([a(AXi)]„^,[C;(Afi)]„^) + ([g(AXi)];37,[e(Ari)];37)]- (74) 

Next we consider each term on the right hand side of (74) separately. By (50e), we have 


~ ~ ft ^ ft— ^ 

(AX^e,Ar^,) =^|-||Ar„,ef + 

1=1 l=ro+l ^ 

While from equations (50a)-(50c) it is not difficult to obtain that 


(75) 


ro ri 


{[g{/\x^)U,[g{^T^)U) = Y, E 

1=1 l'=rQ-\-l 
ro r+1 

([g(AXi)]„^,[a(Afi)]„^) = E E 

1=1 V=ri-\-l 
r\ r+l 

{[5(AV,)|,.„[e(Ari)|,J= Y. 


6 — Oui' 
T^l - ^i' 

0 


6ui 


vi - uu 


■||[e(Ari)],,„,f, 


ll[e(Ari)],,„,| 


/=ro+l /'=riH-l 

Adding the last three equalities together and making suitable rearrangement yields that 

2[([g(AXi)]„;3, [g{Afi)Up) + ([c;(AXi)]„.„ [g(Ari)]„.,) + ([^(axi)];,^, [^(Ari)];,^)] 

ro r+l ^ 

= -2E E 


/=1 /'=ro+l 
ri ro 




ll[e(Ari)],,<,,| 


r+l 


■2 E (Ei^iiie(^L+,.„ip+ E ^mAh)u..f). 


(76) 


r%ll(e(Af.)U,„„|p+ E : 

F=ro+l (=1 ^ ^ F=ri + 1 

In addition, by using equation (50d), we can calculate that 

(i^(AXi),i^(Ari)) = (^(AX„«),i^(Af„«)) + 2([i^(AXi)]„;3,[7^(Afi)]„^) 

+ (i^(AX^^), i^(Af^^)) + 2([7^(AXi)]„^, [7^(Af !)]„.,) + 2([7^(AXi)]^.„ [iW(Af i)]^.,) 

|2 


ro ro 


|^ll[«(Ar.)U.„|p + 2 E E ^lllw(Ari)l 

/=! Z'=l ‘ ‘ i=ro+l/'=ro+l 


2 EE: 


CLiai, I 


ro ri 


+ 2E E ^C5^lll«(Ar,)la,.„lin2E E ^l|[«(Ar,)l 

l=l l'=ro+l l=l Z'=ri+1 ‘ ‘ 


ro r+l 


9 


aiaif I 


ri r+l 


+ 2 E E +;-||[«(Af.)l 


/=ro+l /'=ri+l 
ro r+l 


U + 17;/ 


a-ia.li \ 


ft ~ ft— ~ 

2EE5-n=-ll[«(Ar.)U.„ll^ + 2 E E+Llllw(Ar,)la,.„l 


1=1 V=1 


^=ro+l l'=l 


(77) 
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Substituting equations (75)-(77) into equation (74) immediately yields that 


{{At,AX),iAC,AT)) = {XiAr),AT) 


ro r+1 


e 


ro r+1 


2EE^^IIIw(^rou,P-2j: ^_|||e(Arou, 


1=1 l'=l 
ro 


l=l l'=ro+l 


I'v - 


n ri r+1 

l=l ' 


Vl! + V 
l=ro+ll'=l ‘ 

■2 E [E^|^ii[e(AFi)].,.,ir^+ E 


+ E 

/=ro+l 


ri ro 

duv 


Oul 


V-Vl 




(78) 


/'=ro+l /=1 /=ri+l 

On the other hand, by (60) and the definitions of B and P in Lemma 2.6, we have that 


ro 


c tr [i 3 (AL) {B{X) - V,I)^B{Ar)] P, 


i=i 

ro r+1 


_ _ a 1 '» a ^0 i'+l r\ 


l=l V=ro+l 


l=l 


1=1 V=1 


Vl + UV 


and 


E^fl(r),pjB(Ar)(S(x) -F/)ts(Ar)Pj 


E Es^iii«(^b+..,ir^-i E 


l=ro+l l'=l 
ri 


2 ^ V 

i=ro+l 

r+1 


+ E (E^II[5(Ab)la«,IP+ E ^lll5(Afi+,a„| 

V=ro+l 1=1 '■ l=ri+l '■ 


By substituting the last two equalities into the expression of ((<^, L), (A^, AL)) and 

then comparing with equation (78), we obtain equation (19c). 

Subcase 3.2: ak = 0. Let tq G {1,..., r} be such that a = IJ[=i /3 = IJ[=ro+i 

where a and (3 are defined by (38) with /cq G [0,A; —1] such that equation (37) holds. 
Also, equation (63) holds. By (72) and Lemma 4, equations (51a)-(52c) hold. 

We first prove that equation (19a) holds. By equations (51a)-(51c), the definition of 
the operator D in (45), and equation (47) with C 2 given by (48), we have that 

(AC, P>(Ar)) = (ci>o(T, D{H)), 4)i(r, D{H )),..., ch,+i(T, D{H))) = nc,(r, D{H)). (79) 

By Lemma 5, (AC, AL) G Ck{X). i.e., (19a) follows. By the Moreau decomposition [11], 

{At + AC, D{AX+Af)) = (AC, £>(Ar)) + {At + AC, D{AX + AL)), 
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which implies that {At, D{AX)) = 11^0 (At + AQ,D{AX + AF)). Notice that 

+ TJ^=l{G{^Xa,a^)MKa^)) + ([AXfefe Z^c], [AAfe, Zbc]) 

= {{At,D{AX)),{T,D{Z))) <t), 

for any (r, Z) S Tk^J, X) n (C, F)-*-, where the inequality is using Lemma 5. By Case (ii) 
of Lemma 6, {At, AX) — (0, 17X(AF)1f''^) G [Ck{X)]°. Thus, equation (19b) holds. 

We next prove that (19c) holds. By (At, D{AX)) = nc|(At + A^", D{AX + AF)), 

AtAc + ELi([^(aai)],,,,, [6;(Ari)],,,,) + {[ax^^ ax^,], [ax^^ af^,]) 

= ((At, D{AX)), (AC, T>(Af))) = {Uco (r, D{H)),Uc, (r, D{H))) = 0. 


Together with the arguments for Case (ii) of Lemma 6 and equation (51g), we know that 


((At,AA),(AC,AF)) = (X(AF),AF) 

= 2([g(AXi)]„^,[e(Afi)]„^) + {n{AX^^),n{Af^^)) 
+ 2{[n{AXi)]^i3, [n{Afi)U) + {AX^„ Af„,). 


(80) 


We next consider each term on the right hand side of (80) separately. By (51f), we have 


2([a(AAi)],^,[g(AFi)]„^) 

ro r +1 „ r +1 tq 

=2E E |-ilie(^r.)u.„ip-2 53^|||e(Ar.)u 

7_1 7/_I 1 ^ 7/_I 1 7_1 ^ 


while from equations (52a) and (52b) it is not difficult to obtain that 


{'H{AX^^),'H{AT^^)) + 2{[H{AXi)]^p, [n{ATi)]^p) 



In addition, by using (52c), we can easily obtain that {AXac, AFq,c) = X][=i ^11 AF^jc 
Substituting the three equalities into equation (80) then yields that 


((ALAA),(AC,AF)) 
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On the other hand, by using = —6, X = [/[Diag((7) 0]!/''^ and the definitions of B and 
P in Lemma 2.6, we calculate that 


ro 


2C [B{AT){B{X) - VjI)^B{AT)]Pa^ 

j=i 

>'0 r+l „ 


ro r+l 


e 


ro 


2E E ^ll[e(Ar.)|„„„||=-2j;j;=^||[«(Af.)U,.„| 

l=l V=rQ+l ‘ l=l V=l ‘ ' 


e 


/ = 1 ^ 


aid 


Notice that , ['H(AXi)]a;aj,) = 0 for / e {1,... ,ro} and I' G {rofl, • • • ,r+l}. 

By this, we may calculate that 

2 ^ [s^^(r) 0], [uJaaX^aaF^ uJaxa^aaFs] ^ 

r+l ro r+l ro 

l'=rQ-\-l 1=1 //=ro+l ^=1 


Substituting the last two equalities into the expression of ((^, L), (A<^, AL)) and 

then comparing with equation (81), we obtain equation (19c). 

Conversely, suppose that (19a) and (19c) hold. By Lemma 2.4 it suffices to show that 
equation (72) holds. If d:” G intiL, then {At, AX) = (0,0) since [Ck{X)]° = {(0,0)}. 
If d:” G intiL°, then (A((',Ar) = (0,0) since Ck{X) = {(0,0)}. Together with the 
directional derivative of Yix in the two cases, we obtain equation (72). Next we assume 
that X ^ int TL U int K°. Then it must hold that 

{{At, AX) - X((AC, AT)), (AC, AT)) = 0, (82) 

where X : X ^ X is defined by X{uj,W) := (0, Z7X([7^W)F^) for any (w, IT) G X, 
Indeed, from the previous arguments in Subcase 3.1 and Subcase 3.2, we know that 

(x((Ac,Ar)),(Ac,Ar)) = ((o,c7x(Af)F^),(AC,Ar)) = -T(j^^)((c,r),(Ac,Ar)) 

= -u((c,r),r|((t,A),(Ac,Ar))). (83) 

Along with (19c), we obtain equality (82). Combining (82) with (19a)-(19b) yields that 





G ^CkX) 



Together with (83), using [2, Theorem 7.2] with d = (At + AC, AX + AT) and u* = 
(AC, AT) shows that equation (72) holds. The proof is then completed. □ 
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